The present invention relates to a picture processor for deforming a picture to be displayed on a display of a personal computer or the like.
Moreover, the present invention relates to a picture data processor for processing the picture data for a plurality of pictures when displaying the pictures on a display.
Furthermore, the present invention relates to a variable-length-code decoder for variable-length-decoding a plurality of variable-length-encoded MPEG picture data values.
(1) Orthogonal-transformation-system encoded picture data such as MPEG or DV has been used so far as digital data serving as an original picture for computer graphics to be displayed on a display of a personal computer or the like.
There is a mapping apparatus for decoding the encoded picture data such as MPEG and DV shown in FIG. 7(a), deforming a picture 15 obtained by decoding the encoded data, and mapping the deformed picture on, for example, a room wall 17 formed through computer graphics on a display 16 of a personal computer or the like as shown in FIG. 7(b).
Hereafter, an MPEG picture mapping apparatus for mapping a picture obtained from the MPEG picture data and a DV picture mapping apparatus for mapping a picture obtained from the DV picture data are described below in order by referring to the accompanying drawings.
First, a conventional MPEG-picture mapping apparatus is described.
FIG. 8 shows a block diagram of the conventional MPEG-picture mapping apparatus.
VLD means 1 inputs and variable-length-decodes variable-length-encoded MPEG picture data, inverse-quantization means 2 inversely quantizes the variable-length-decoded MPEG picture data, and IDCT means 3 decodes the inversely-quantized MPEG picture data. When the IDCT means 3 decodes the inversely-quentized MPEG picture data, it decodes all MPEG picture data values every block of an MPEG picture having a plurality of 8xc3x978 DCT coefficients.
When the MPEG picture data input by the VLD means 1 is the data for an in-frame encoded picture (hereafter referred to as I-picture), the IDCT means 3 outputs a decoded picture to format transformation means 12 by passing the picture through addition means 7 and moreover, outputs the picture to a frame buffer 8. The frame buffer 8 stores decoded pictures supplied from the IDCT means 3.
However, when the MPEG picture data input by the VLD means 1 is the data for an inter-frame forward-prediction encoded picture (hereafter referred to as P-picture) or the data for an inter-frame bidirectional-estimation encoded picture, the frame buffer 8 already inputs an I-picture once or already stores the I-picture or a P-picture perfectly-decoded picture in which a movement vector and a difference picture are added to the I-picture. In this case, the IDCT means 3 outputs a movement vector supplied from the inverse-quantization means 2 to movement compensation means 9 through the frame buffer 8 and the movement compensation means 9 generates a movement-compensated picture in which only a movement vector is compensated by adding only the movement vector to a perfect picture stored in the frame buffer 8. Then, the addition means 7 adds the movement-compensated picture supplied from the movement compensation means 9 and a decoded difference picture supplied from the IDCT means 3 each other and outputs an added picture to the format transformation means 12 and also, outputs the picture to the frame buffer 8. Moreover, the frame buffer 8 stores perfect decoded pictures supplied from the addition means 7.
Then, the format transformation means 12 transforms a YUV-format picture supplied from the IDCT means 3 or addition means 7 into an RGB-format picture and outputs the RGB-format picture to picture deformation means 18.
The picture deformation means 18 inputs the RGB-format picture from the format transformation means 12 and moreover inputs the information for the degree of enlargement or contraction of each portion of the picture from CG original-data generation means 4 through deformation information generation means 5, deforms the picture supplied from the format transformation means 12 by interpolating or thinning pixels of each portion of the picture in accordance with the information for the degree of enlargement or contraction, and outputs the deformed picture to mapping means 13.
The mapping means 13 maps a picture supplied from the picture deformation means 18 to a predetermined position on the display of a personal computer or the like in accordance with the mapping-position information of a picture supplied from the CG original-data generation means 4.
Then, a conventional DV-picture mapping apparatus is described below.
FIG. 9 shows a block diagram of the conventional DV-picture mapping apparatus.
VLD means 1 inputs and variable-length-decodes variable-length-encoded DV picture data, inverse-quantization means 2 inversely quantizes the variable-length-decoded DV picture data, and IDCT means 3 decodes the inversely-quantized DV picture data. When the inversely-quantized DV picture data is decoded, the IDCT means 3 decodes all DV picture data values every block of the DV picture having a plurality of 8xc3x978 DCT coefficients. Moreover, the IDCT means 3 outputs partial pictures decoded every block to deshuffling means 14.
The partial pictures decoded every block output by the IDCT means 3 are not orderly output like the following: for example, from the top block of the leftmost column of the entire picture constituted of the partial pictures to the bottom block and moreover, from the top block of the second column from the top left to the bottom block. That is, the sequence of partial pictures input by the deshuffling means 14 is shuffled. Therefore, the deshuffling means 14 deshuffles the partial pictures supplied from the IDCT means 3, orderly arranges them, and generates a perfect DV picture.
Then, format transformation means 12 transforms a YUV-format picture supplied from the deshuffling means 14 into an RGB-format picture and outputs the RGB-format picture to picture deformation means 18. Subsequent DV-picture operations are the same as those of the above-described MPEG-picture mapping apparatus.
As described above, as for the conventional MPEG-picture mapping apparatus and DV-picture mapping apparatus, a picture input by the format transformation means 12 is a full-size, picture in which pixels of the picture are interpolated or thinned every portion of the picture in accordance with the information for the degree of enlargement or contraction of each portion of the picture in the picture deformation means 18.
However, it is preferable that the picture input by the format transformation means 12 is a picture deformed under decoding in accordance with the information for the degree of enlargement or contraction of each portion of the picture.
It is an object of the first aspect of the present invention to provide a picture processor for deforming a picture in accordance with the information for the degree of enlargement or contraction of each portion of the picture when decoding encoded picture data by considering the problem that the above picture deformation was not performed conventionally when decoding the encoded picture data.
(2) Moreover, as described above, it has been more important in recent years to reproduce MPEG picture data and display a picture on the display of a personal computer or the like. The throughput for decoding or the like until displaying the MPEG picture data as a picture is very large. Therefore, an apparatus for processing the MPEG picture data can only process one MPEG picture data value at one time.
A conventional method for decoding MPEG picture data and displaying a picture on the display window of a personal computer is described below. Because there are two types of conventional methods, they are described separately.
The first method is a method for completely decoding MPEG picture data to reproduce a picture, interpolating or thinning pixels so as to fit the picture to the size and shape of a window, and displaying the picture on the window.
The second method is a method for selecting all or some of DCT coefficients of MPEG picture data in accordance with the size and shape of a window before decoding the MPEG picture data and directly decoding the DCT coefficients or decoding all the DCT coefficients by adding a predetermined number of zeros to a high-frequency region, and displaying the decoded picture on a window by interpolating or thinning pixels so as to fit the picture to the size and shape of the window.
Moreover, a window system has been spread which decodes a plurality of MPEG picture data values and simultaneously displays a plurality of pictures on the display of a personal computer or the like. For example, a window system is used which displays a picture supplied from a DVD on the display of a personal computer and simultaneously displays a picture supplied from a CD-ROM on the same display.
As described above, to display a plurality of pictures on the same display at the same time, the above first or second method prepares a picture data processor for each picture and reproduces and displays the picture so as to fit the picture to the size and shape of each window. These conventional picture data processors have a throughput capable of forming a large picture even if a picture to be formed is very small.
As described above, a conventional picture data processor cannot simultaneously process two MPEG picture data values or more.
Therefore, to reproduce a plurality of MPEG picture data values and display a plurality of pictures on the display of a personal computer or the like at the same time, picture data processors equal to the number of pictures to be displayed are necessary. That is, picture data processors equal to the number of pictures, respectively corresponding to the size and shape of each picture to be displayed and respectively having a large throughput are necessary independently of the amount of MPEG picture data to be processed.
It is an object of the second aspect of the present invention to provide a picture data processor capable of flexibly processing a plurality of MPEG picture data values even when reproducing a plurality of MPEG picture data values and simultaneously displaying a plurality of pictures on the display of a personal computer or the like by considering the problem that picture data processors equal to the number of pictures and respectively having a large throughput are conventionally necessary when reproducing a plurality of MPEG picture data values and displaying a plurality of pictures on the display of a personal computer or the like.
(3) Moreover, MPEG picture data values supplied from a plurality of data streams are decoded to display a plurality of pictures on the display of a personal computer or the like. Variable-length-encoded data values are included in the MPEG picture data values and the variable-length-encoded data values are first variable-length-decoded and then decoded into pictures.
A conventional variable-length-code decoder for variable-length-decoding a plurality of variable-length-encoded MPEG picture data values like the above mentioned is described below by referring to the accompanying drawings.
FIG. 26 shows a block diagram of a conventional variable-length-code decoder. The conventional variable length-code decoder is constituted of a first variable-length-data decoding circuit 5400, a second variable-length-data decoding circuit 5500, a third variable-length-data decoding circuit 5600, . . . , and an nth variable-length-data decoding circuit 5700 and each of the variable-length-data decoding circuits has a shift circuit 58 and an encoded-word decoding circuit 59.
The format of a picture of data is analyzed, for example, it is checked by a first MPEG-data hierarchical-structure decoding circuit 1099 whether the variable-length-encoded MEG picture data supplied from a first picture-code generation source 600 is constituted of a movement vector and difference data and the format is input to the first variable-length-data decoding circuit 5400.
Then, the first variable-length-data decoding circuit 5400 variable-length-decodes variable-length-encoded MPEG picture data and outputs the data to an information-source decoder group 1400. The variable-length decoding is described later.
Similarly, each of the formats of variable-length-encoded MPEG picture data values supplied from a second picture-code generation source 700, a third picture-code generation source 800, . . . , and an nth picture-code generation source 900 is analyzed by a second MPEG-data hierarchical-structure decoding circuit 1100, a third MPEG-data hierarchical-structure decoding circuit 1200, . . . , and an nth MPEG-data hierarchical-structure decoding circuit 1300 and input to the second variable-length-data decoding circuit 5500, third variable-length-data decoding circuit 5600, and nth variable-length-data decoding circuit 5700.
Then, each of the second variable-length-data decoding circuit 5500, third variable-length-data decoding circuit 5600, . . . , and nth variable-length-data decoding circuit 5700 variable-length-decodes variable-length-encoded MPEG picture data and outputs the data to the information-source decoder group 1400.
Thereafter, the information-source decoder group 1400 decodes the variable-length-decoded MPEG picture data supplied from each of the first variable-length-data decoding circuit 5400, second variable-length-data decoding circuit 5500, third variable-length-data decoding circuit 5600, . . . , and nth variable-length-data decoding circuit 5700 to generate a picture corresponding to the data supplied from each variable-length-data decoding circuit.
Then, variable-length decoding performed by each variable-length-data decoding circuit is further described below.
For the description, FIG. 6 shows an example of variable-length-encoded MPEG picture data input by a variable-length-data decoding circuit. Symbols A, B, C, and D in FIG. 6 denote n-, m-, l-, and o-bit variable-length-encoded words respectively, which are assumed to be continues.
The shift circuit 58 of a variable-length-data decoding circuit inputs a word length-capable of sufficiently storing the maximum word length of variable-length-encoded MPEG picture data and temporarily stores the word length. To simplify description, it is assumed that the data to be temporarily stored is constituted of the encoded words A, B, C, and D in FIG. 6.
Because the number of bits of a code length of each encoded word is unknown, the encoded word decoding circuit 59 detects the foremost encoded word of the variable-length-encoded data input by the shift circuit 58 and the length of the encoded word by using a uniquely-separable table. That is, the encoded word decoding circuit 59 detects the encoded word A in FIG. 6 and that the encoded word A has a word length of n bits.
Moreover, the encoded word decoding circuit 59 variable-length-decodes the foremost encoded word and outputs the variable-length-decoded data to the information-source decoder group 1400 and moreover, outputs the information for the number of bits of the code length of the output data to the shift circuit 58. That is, the encoded word decoding circuit 59 variable-length-decodes the encoded word A and outputs it to the information-source decoder group 1400 and moreover outputs the information that the output data had a word length of n bits to the shift circuit 58.
Thereafter, the shift circuit 58 receives the variable-length-encoded data equivalent to the word length from an MPEG-data hierarchical-structure decoding-circuit in accordance with the information for the word length of the output data, for example, n bits, and shifts the data temporarily stored by, for example, n bits. The head of the data temporarily stored in the shift circuit 58 due to the above shift serves as the head of the next encoded word, that is, the head of the encoded word B in FIG. 6.
Thus, a variable-length-data decoding circuit successively detects the encoded word of variable-length-encoded data and variable-length-decodes it.
However, a conventional variable-length-code decoder has disadvantages that it requires a plurality of variable-length-data decoding circuits and has a too large circuit, because it variable-length-decodes each of a plurality of variable-length-encoded MPEG picture data values for every data.
Therefore, the following apparatus is assumed as a variable-length-code decoder solving the disadvantage of a too large circuit.
The apparatus is provided with only one variable-length-data decoding circuit. That is, the apparatus is provided with the shift circuit 58 and the encoded word decoding circuit 59 one each.
The shift circuit 58 receives each of variable-length-encoded MPEG picture data values supplied from a first MPEG-data hierarchical-structure decoding circuit 6, a second MPEG-data hierarchical-structure decoding circuit 7, a third MPEG-data hierarchical-structure decoding circuit 8, . . . , and an nth MPEG-data hierarchical-structure decoding circuit 9 by means of time sharing. Moreover, the encoded word decoding circuit 59 variable-length-decodes and outputs the foremost encoded word in a data stream supplied from one of a plurality of MPEG-data hierarchical-structure decoding circuits through the above-described operation. Moreover, the shift circuit 58 shifts the stream of the output data by a value equivalent to the word length of the data output from the encoded word decoding circuit 59 as described above. In this case, no processing is applied to data streams other than the processed data stream. Thereafter, successively similarly, the encoded word decoding circuit 59 variable-length-decodes and outputs the foremost encoded word of a data stream supplied from other MPEG-data hierarchical-structure decoding circuit and the shift circuit 58 shifts the stream of output data by a value equivalent to the word length of the output data.
However, the above assumed variable-length-code decoder has a disadvantage that the variable-length decoding speed is too low though it conquers the disadvantage of a too large circuit. That is, the encoded word decoding circuit 59 has a disadvantage that it does not process the data supplied from other MPEG-data hierarchical-structure decoding circuits while it processes the data supplied from one MPEG-data hierarchical-structure decoding circuit.
It is an object of the third aspect of the present invention to provide a variable-length-code decoder having a small circuit and capable of performing variable-length decoding at a high speed by considering a problem of avoiding a too large circuit and a problem of improving the variable-length-decoding speed.
The first aspect of the present invention provides a picture processor comprising picture-deformation-information input means for inputting the information for the degree of enlargement or contraction of each portion of a picture; decoding means for inputting the picture data in which the picture is encoded and moreover inputting the information for the degree of enlargement or contraction of each portion of the picture from the picture-deformation-information input means, decoding the picture data for each portion of the picture in accordance with the information showing the degree of enlargement or contraction and moreover, decoding the entire picture data independently of the information showing the degree of enlargement or contraction; reference-picture generation means for generating a reference picture in which each portion of the picture is enlarged or contracted by using a picture in which the entire picture data supplied from the decoding means is decoded, the information showing the degree of enlargement or contraction of each portion of the picture supplied from the picture-deformation-information input means, and the movement vector of the picture data; and addition means for adding the picture in which each portion of the picture is decoded supplied from the decoding means and the reference picture supplied from the reference-picture generation means.
Moreover, the third aspect of the present invention provides a picture processor comprising picture-deformation-information input means for inputting the information showing the degree of enlargement or contraction of each portion of a picture and decoding means for inputting the picture data in which the picture is encoded and moreover inputting the information for the degree of enlargement or contraction of each portion of the picture from the picture-deformation-information input means and decoding the picture data for each portion of the picture in accordance with the information for the degree of enlargement or contraction.
The second aspect of the present invention provides a picture data processor comprising enlargement/contraction-information input means for inputting the degree of enlargement or contraction of a plurality of picture data values, picture processing means having a plurality of processing units for processing picture data to input a plurality of picture data values and process pictures, and assignment means for inputting the information for the degree of enlargement or contraction from the enlargement/contraction-information input means and assigning all or some of the processing-unit groups of the picture processing means to the enlargement or contraction processing of the picture data corresponding to the information for the degree of enlargement or contraction in accordance with the information for the degree of enlargement or contraction, wherein the picture processing means processes a plurality of picture data values in accordance with the information for the degree of enlargement or contraction and the assignment of the processing-unit groups.
The third aspect of the present invention provides a variable-length-code decoder comprising input means for successively continuously inputting variable-length-encoded MPEG picture data from a plurality of streams having the variable-length-encoded MPEG picture data by means of time sharing so that the data quantity supplied from each stream increases up to a predetermined quantity and variable-length decoding means for inputting the predetermined quantity of MPEG picture data from the input means, variable-length-decoding the data, outputting a predetermined decoded word to the outside of the variable-length-code decoder, and outputting the information for the length of the decoded word to be output to the input means, wherein the variable-length decoding means inputs the predetermined quantity of MPEG picture data from every stream other than the stream of the MPEG picture data for the decoded word to be output before outputting the predetermined decoded word to the outside of the variable-length-code decoder, and the input means inputs MPEG picture data from the stream of the MPEG picture data of the decoded word finally output to the outside of the variable-length-code decoder by the variable-length decoding means by using the information for the length of the output decoded word before the variable-length decoding means outputs the predetermined decoded word to the outside of the variable-length-code decoder and outputs a predetermined decoded word among the MPEG picture data of the stream next to the stream of the MPEG picture data of the output decoded word.
A high-speed shift circuit 200 serving as the input means of a variable-length-code decoder of the third aspect of the present invention is able to shift data at a high speed compared to the shift circuit 58 of the conventional variable-length-code decoder. Therefore, the variable-length decoding means of a variable-length-code decoder of the present invention is able to output a predetermined decoded word to the outside of the variable-length-code decoder and the input means is able to shift and input the MPEG picture data equivalent to the length of the encoded word before the decoded word is decoded from the stream of the output decoded word.